3,137 research outputs found
Towards a Learning Theory of Cause-Effect Inference
We pose causal inference as the problem of learning to classify probability
distributions. In particular, we assume access to a collection
, where each is a sample drawn from the
probability distribution of , and is a binary label
indicating whether "" or "". Given these data,
we build a causal inference rule in two steps. First, we featurize each
using the kernel mean embedding associated with some characteristic kernel.
Second, we train a binary classifier on such embeddings to distinguish between
causal directions. We present generalization bounds showing the statistical
consistency and learning rates of the proposed approach, and provide a simple
implementation that achieves state-of-the-art cause-effect inference.
Furthermore, we extend our ideas to infer causal relationships between more
than two variables
Unified Uncertainty Calibration
To build robust, fair, and safe AI systems, we would like our classifiers to
say ``I don't know'' when facing test examples that are difficult or fall
outside of the training classes.The ubiquitous strategy to predict under
uncertainty is the simplistic \emph{reject-or-classify} rule: abstain from
prediction if epistemic uncertainty is high, classify otherwise.Unfortunately,
this recipe does not allow different sources of uncertainty to communicate with
each other, produces miscalibrated predictions, and it does not allow to
correct for misspecifications in our uncertainty estimates. To address these
three issues, we introduce \emph{unified uncertainty calibration (U2C)}, a
holistic framework to combine aleatoric and epistemic uncertainties. U2C
enables a clean learning-theoretical analysis of uncertainty estimation, and
outperforms reject-or-classify across a variety of ImageNet benchmarks. Our
code is available at:
https://github.com/facebookresearch/UnifiedUncertaintyCalibratio
Non-linear Causal Inference using Gaussianity Measures
We provide theoretical and empirical evidence for a type of asymmetry between
causes and effects that is present when these are related via linear models
contaminated with additive non-Gaussian noise. Assuming that the causes and the
effects have the same distribution, we show that the distribution of the
residuals of a linear fit in the anti-causal direction is closer to a Gaussian
than the distribution of the residuals in the causal direction. This
Gaussianization effect is characterized by reduction of the magnitude of the
high-order cumulants and by an increment of the differential entropy of the
residuals. The problem of non-linear causal inference is addressed by
performing an embedding in an expanded feature space, in which the relation
between causes and effects can be assumed to be linear. The effectiveness of a
method to discriminate between causes and effects based on this type of
asymmetry is illustrated in a variety of experiments using different measures
of Gaussianity. The proposed method is shown to be competitive with
state-of-the-art techniques for causal inference.Comment: 35 pages, 9 figure
Structural Agnostic Modeling: Adversarial Learning of Causal Graphs
A new causal discovery method, Structural Agnostic Modeling (SAM), is
presented in this paper. Leveraging both conditional independencies and
distributional asymmetries in the data, SAM aims at recovering full causal
models from continuous observational data along a multivariate non-parametric
setting. The approach is based on a game between players estimating each
variable distribution conditionally to the others as a neural net, and an
adversary aimed at discriminating the overall joint conditional distribution,
and that of the original data. An original learning criterion combining
distribution estimation, sparsity and acyclicity constraints is used to enforce
the end-to-end optimization of the graph structure and parameters through
stochastic gradient descent. Besides the theoretical analysis of the approach
in the large sample limit, SAM is extensively experimentally validated on
synthetic and real data
- …